From this data we can see that our variables have a variety of different values based on their types. carat has a mean of 0.79 but max of 4.1. We see several variables having a wide range of values, most noticeably price with a range from 336 to 18823, or width with a range of 0 to 9.94 There may be high correlation between the variables as well, for example depth and depth_perc, so we probably will remove one in the analysis. For our target variable cut, the value is High if the cut if Premium or Ideal, and Acceptable if the value is Fair, Good, or Very Good.

Now we can see the range of values for each variable.

• We can see that about 69% of the data are categorized as ‘High’ in cut quality. Looking at the potential relationship, we can see the strongest relationships are with carat and length.

• We see the largest concentration of diamonds’ price around $0-$5,000. The data is also skewed to the right. Looking at potential relationships, we can see strong relationships between price and carat, length, width, and depth, suggesting these variables have impacts on the price of a diamond.

• The higher than average relationship between certain variables (example: width and leghth) may be sign of multicollinearity and we will probably address this later in the analysis.

Here is a look at the training and testing dataset

Here is a look at a regression model predicting price

Here is a look at a logistic regression model predicting diamond cut

• For this part of the analysis, I perform a Tuned Bootstrap Aggregrating (Bagged) model to predict the price.
• For the CV folds, I want 5-fold cross validation, so that the training data is split into 5 equally sized folds. I set the number of trees to 500 in the random forest, and the model will test the minimum sample split into 5, 15, or 25. Similarly, the model will also test the maximum depth of the trees, with the values of 5, 6, 7, 8.
• I then let R choose the parameter for maximum performance, using the default metrics RSME.

• For this part of the analysis, I perform a Tuned Random Forest model to predict the price.
• For the CV folds, I want 5-fold cross validation, so that the training data is split into 5 equally sized folds. I set the number of trees to 500 in the random forest, and the model will test the minimum sample split into 5, 15, or 25. Similarly, the model will also test the maximum depth of the trees, with the values of 5, 6, 7, 8. The number of variables in the model ranges from 4 to 7.
• I let R choose the parameter for maximum performance, using the default metrics RSME.

• For this part of the analysis, I perform a Random Forest model to predict the cut of a diamond
• Due to performance issue, I couldn’t afford to tune the model. I will assign the number of variables randomly selected for the model (mtry) as 5, number of trees (trees) as 500, minimum number of observations (min_n) as 15, and maximum tree depth (max.depth) as 7.

• For this part of the analysis, we perform a Tuned Gradient Boosted (XGBoost) model to predict the cut of a diamond.
• We use a grid of hyperparameters using a Latin hypercube sampling strategy, where class_xg_grid will contain a dataframe or tibble with 10 rows, where each row represents a unique combination of the specified hyperparameters.

To predict price, here are the most important variables:

• Width
• Carat
• Clarity
• Length
• Color

For the price, how big, bright, and clear the diamond have a big impact!

To predict cut, here are the most important variables:

• Length
• Width
• Depth
• Depth_Perc
• Table_Perc

We can see that the different measurements of the diamond affect the cut of the diamond.

• We can see that compared to Linear Regression models, the Bagged Model and Random Forest Model both perform better. While they have the same R-square, our Tuned Random Forest model has a slightly lower MAE and RMSE. Therefore, I would recommend the Tuned Random Forest model for this analysis.

• For cut prediction, I would recommend using Gradient Boosted model due to the high number of sensitivity, precision, and AUC. The model was able to catch 86% - 97% of the true positive values, and out of the positive values it predicted, 83% - 88% were true (depending on whether you use a best cutoff or not).

• My biggest challenge in this project is that re-categorizing my categorical variables. I have a lot of categories in columns clarity, color, and cut and it was difficult to determine how to re-categorize them so that I only have about 2 to 3 categories in each column.
• Trying to determine the tuning parameters of my Random Forest and XGBoost models also posed a problem, because there are so many parameters to try out.
• What I’m most proud of in this project is that even though I ran into speed issue when I tried to run my tuned models, I still came up with some results that was significant in predicting my target variables.

If I had another week to work on this project, I would love to try out more models we have done in class, such as Lasso and SVC. I would also try to normalize and balance the data set like what we did in homework case #2 and #3, to see if transforming the data set has any impacts on the performance of the models.